Measuring the Effects of Advertising on Sales via a Controlled Experiment on Yahoo!

A randomized experiment performed in cooperation between Yahoo! and a major retailer allows us to measure the effects of online advertising on sales. We exploit a match of over one million customers between the databases of Yahoo! and the retailer, assigning them to treatment and control groups for an online advertising campaign for this retailer and then measuring each individual’s weekly sales at this retailer, both online and in stores. By combining a controlled experiment with panel data on purchases, we find statistically and economically significant impacts of the advertising on sales. The treatment effect persists for weeks after the end of an advertising campaign, and we estimate the total effect on revenues to be more than eleven times the retailer’s expenditure on advertising during the study. Additional results explore differences in the number of advertising impressions delivered to each individual, age and gender demographics, online and offline sales, and the effects of advertising on those who click the ads versus those who merely view them. Our results provide the best measurements to date of the effectiveness of image advertising on sales, and we shed light on important questions about online advertising in particular. * Lewis: Massachusetts Institute of Technology, [email protected]. Reiley: Yahoo! Research and University of Arizona, [email protected]. We thank Meredith Gordon, Sergiy Matusevych, and especially Taylor Schreiner for their work on the experiment and the data. Yahoo! Incorporated provided financial and data assistance, as well as guaranteeing academic independence prior to our analysis, so that the results could be published no matter how they turned out. We acknowledge the helpful comments of Manuela Angelucci, JP Dubé, Kei Hirano, John List, Preston McAfee, Paul Ruud, Michael Schwarz, PaiLing Yin, and seminar participants at University of Arizona, University of California at Davis, New York University, Sonoma State University, Vassar College, the FTC Microeconomics Conference, and Economic Science Association meetings in Pasadena, Lyon, and Tucson.